muzero | simple implementation of MuZero algorithm | Machine Learning library

Author here.

What does the reward from the initial_inference represent?

The initial inference "predicts" the last observed reward. This isn't actually used for anything, but makes our code simpler: The prediction head can simply always predict the immediately preceding reward. For the dynamics network, this would be the reward observed after applying the action that's given as an input to the dynamics network.

At the beginning of the game there is no last observed reward, so we just set it to 0.

The reward target computation in the pseudocode was indeed misaligned; I've just uploaded a new version to arXiv.

Where it used to say